A Kalman Filter Based Framework for Monitoring the Performance of In-Hospital Mortality Prediction Models Over Time (preprint)

Unlike in a clinical trial, where researchers get to determine the least number of positive and negative samples required, or in a machine learning study where the size and the class distribution of the validation set is static and known, in a real-world scenario, there is little control over the size and distribution of incoming patients. As a result, when measured during different time periods, evaluation metrics like Area under the Receiver Operating Curve (AUCROC) and Area Under the Precision-Recall Curve(AUCPR) may not be directly comparable. Therefore, in this study, for binary classifiers running in a long time period, we proposed to adjust these performance metrics for sample size and class distribution, so that a fair comparison can be made between two time periods. Note that the number of samples and the class distribution, namely the ratio of positive samples, are two robustness factors which affect the variance of AUCROC. To better estimate the mean of performance metrics and understand the change of performance over time, we propose a Kalman filter based framework with extrapolated variance adjusted for the total number of samples and the number of positive samples during different time periods. The efficacy of this method is demonstrated first on a synthetic dataset and then retrospectively applied to a 2-days ahead in-hospital mortality prediction model for COVID-19 patients during 2021 and 2022. Further, we conclude that our prediction model is not significantly affected by the evolution of the disease, improved treatments and changes in hospital operational plans.

Do Socialization Restrictions Prevent Restaurants from Becoming Covid Hotspots? (preprint)

Simulation models for infection spread can help understand what factors play a major role in infection spread. Health agencies like the Center for Disease Control (CDC) can accordingly mandate effective guidelines to curb the spread. We built an infection spread model to simulate disease propagation through airborne transmission to study the impact of restaurant operational policies on the Covid-19 infections. We use the Wells-Riley model to measure the expected value of new infections in a given time-frame in a particular location. For the purpose of this study, we have restricted our analysis to bars and restaurants in the Minneapolis-St. Paul region. Our model helps identify disease hotspots within the Twin Cities and proves that stay-at-home orders were effective during the recent lockdown, and the people typically followed the social distancing guidelines. To arrive at this conclusion, we performed significance testing by considering specific hypothetical scenarios. At the end of the study, we discuss the reasoning behind the hotspots, and make suggestions that could help avoid them.

HPV and HBV vaccine hesitancy, intention and uptake in the era of social media and COVID-19: A review (preprint)

Prior to the COVID-19 pandemic, the World Health Organization named vaccine hesitancy as one of the top 10 threats to global health. The impact of hesitancy on uptake of human papillomavirus (HPV) vaccines was of particular concern, given the markedly lower uptake compared to other adolescent vaccines in some countries, notably the United States. With the recent approval of COVID-19 vaccines coupled with the widespread use of social media, concerns regarding vaccine hesitancy have grown. However, the association between COVID-related vaccine hesitancy and cancer vaccines such as HPV is unclear. To examine the potential association, we performed two reviews using Ovid Medline and APA PsychInfo. Our aim was to answer two questions: (1) Is COVID-19 vaccine hesitancy, intention, or uptake associated with HPV or HBV vaccine hesitancy, intention, or uptake? and (2) Is exposure to COVID-19 vaccine misinformation on social media associated with HPV or HBV vaccine hesitancy, intention, or uptake? Our review identified few published empirical studies that addressed these questions. Our results highlight the urgent need for studies that can shift through the vast quantities of social media data to better understand the link between COVID-19 vaccine misinformation and disinformation and its impact on uptake of cancer vaccines.

Continual Deterioration Prediction for Hospitalized COVID-19 Patients (preprint)

Leading up to August 2020, COVID-19 has spread to almost every country in the world, causing millions of infected and hundreds of thousands of deaths. In this paper, we first verify the assumption that clinical variables could have time-varying effects on COVID-19 outcomes. Then, we develop a temporal stratification approach to make daily predictions on patients' outcome at the end of hospital stay. Training data is segmented by the remaining length of stay, which is a proxy for the patient's overall condition. Based on this, a sequence of predictive models are built, one for each time segment. Thanks to the publicly shared data, we were able to build and evaluate prototype models. Preliminary experiments show 0.98 AUROC, 0.91 F1 score and 0.97 AUPR on continuous deterioration prediction, encouraging further development of the model as well as validations on different datasets. We also verify the key assumption which motivates our method. Clinical variables could have time-varying effects on COVID-19 outcomes. That is to say, the feature importance of a variable in the predictive model varies at different disease stages.